Time management in a Poisson fishing model 
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Abstract. The aim of the paper is to extend the model of "fishing problem". 
The simple formulation is following. The angler goes to fishing. He buys fishing 
ticket for a fixed time. There are two places for fishing at the lake. The fishes 
are caught according to renewal processes which are different at both places. The 
fishes' weights and the inter-arrival times are given by the sequences of i.i.d. random 
variables with known distribution functions. These distributions are different for 
the first and second fishing place. The angler's satisfaction measure is given by 
difference between the utility function dependent on size of the caught fishes and 
the cost function connected with time. On each place the angler has another utility 
functions and another cost functions. In this way, the angler's relative opinion about 
these two places is modeled. For example, on the one place better sort of fish can be 
caught with bigger probability or one of the places is more comfortable. Obviously 
our angler wants to have as much satisfaction as possible and additionally he have 
to leave the lake before the fixed moment. Therefore his goal is to find two optimal 
stopping times in order to maximize his satisfaction. The first time corresponds to 
the moment, when he eventually should change the place and the second time, when 
he should stop fishing. These stopping times have to be less than the fixed time of 
fishing. The value of the problem and the optimal stopping times are derived. 
Keywords, fishing problem, optimal stopping, dynamic programming, semi-Markov 
process, infinitesimal generator 



1 Introduction 



The solution of double optimal stopping problem, in so called " fishing prob- 
lem", will be presented. One of t he fir st author who considered the basic ver- 
sion of this problem was IStard (11974 and further generalizations we r e don e 



by ^Sta rr and Woo droofd (|l974l ). IStarr. Wardrop. and Woodroofd (|l976l) . 
Kramer and Staril (|l99Cll ). The detail ed review o f the papers connected with 



"fishing problem" was presented by iFergusonI (|l997l ). The simple formula- 



tion of our double optimal stopping problem is following. The angler goes to 
fishing. He buys fishing ticket for a fixed time tg. There are two places for 
fishing at the lake and he can change the place at any moment s. The fishes 
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are caught according to renewal processes {Ni{t),t > 0}, where Ni{t) is the 
number of claims during the time t at the place i e {1, 2}. Let T^.n denote the 
moment of n-th claim at the place i (we fix Ti_o = and T2fi = s), then the 
random variables Si^n = — ?i,n-i are i.i.d. with continuous distribution 
function (c.d.f. for short) Fi. The fishes' weights, which were caught at the 
place i, are given by the sequence of i.i.d. random variables Xi^Q, Xi_2, ■ ■ ■ 
with c.d.f. Hi (we fix Xi^ — and X2.0 — 0). The renewal process is indepen- 
dent on the sequence of claims. The angler's satisfaction measure is given by 
difference between the utility function gi : [0, 00) — > [0, Gi] dependent on size 
of the caught fishes and the cost function q : [0, ^o] [0, Ci] connected with 
time. We assume that gi and Ci are continuous and bounded, additionally 
Ci is differentiable. On each place the angler has different utility functions 
and cost functions. In this way, the angler's relative opinion about these two 
places is modeled. For example, on the one position better sort of fish can be 
caught with bigger probability or one of the piers is more comfortable. The 
angler can change the place of fishing at any time s. The mass of the fishes 
which were caught up to time t if the change of the position took place 
at the time s {Mt — M|) is given by 

Ni(s/\t) N2((t-s) + ) 

n—1 n—1 

Let Z{s,t) denote the angler's payoff for stopping at time t if the change of 
the position took place at time s. The payoff can be expressed as 

( gi{Mt) - ci{t) iit<s<ta, 
Z{s, t) = <^ 51 (M,) - ci(s) + g2{M^ - M,) ~ C2{t ~ s) ifs<t<to, (1) 
[ -C if to < t, 

where C — Ci + C2. With the notation W2{m, s,m^t) = wi{m,s) + g2{fn — 
fn) — C2{t — s) and wi{m,t) — gi{rn) — ci{t), formula ([1]) reduces to 

Z(s, t) = l{t<s<to}MMt, t) + I{.<t<to}W2(M„ s, M|, t) - l{t,<t}C (2) 
2 The optimization problem 

Let Tt = cr(Xi,o,ri,o,^i4,Ti4, . . . ,Xi^jVi(t),7'i,JVi(t)) be the cr-field gener- 
ated by all events up to time t, if there was no change of parameters and 
Ts,t = '^i^s,X2fi,T2fi, . . . ,X2,N2{t-s),T2^N2(t-s)), the cr-field generated by 
all events up to time t if the change of parameters was at time s. For sim- 
plicity of notation we set Tn ■= J-Ti „, J^s.n ■— ^s,T2 „• Let Ai{Tn) denote 
the set of non-negative and ^„-nicasurable random variables. From now on, 
T and stands for the sets of stopping times with respect to the cr-fields 
{^ti i > 0} and {J-s,t^ < s < t}, respectively. Furthermore, define for n G N 
and n < K the sets Tnji ^ {t & T : r > 0, ri_„ < r < Ti,a'} and 
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X — {t ^ : Q < s < T, T2^n < t < T2^k}- Obviously our angler wants 
to have as much satisfaction as possible and he has to leave the lake before 
the fixed moment. Therefore his goal is to find two stopping times rj* and r| 
such that the expected gain is maximized 

EZ{t*, T2) ^ snp sup EZ{ti,T2), (3) 

where rj* corresponds to the moment, when he eventually should change the 
place and r|, when he should stop fishing. These stopping times should be 
less than the fixed moment Iq. The process Z{s,t) is piecewise-deterministic 
and belongs to the class of semi-M arkov processes. The optima l stopp ing of 
similar processes was studied by iBoshuizen and Gouweleeuw ( 1993 ). The 



special representation of stopping times from T and is applied. It allows 
to use the dynamic programming methods to find these two optimal stopping 
times and to specify the expected satisfac tion of the angler. The way of th e 
solution is similar to the methods used by iKarpowicz and Szaiowski ( 2007 ). 



Let us first observe that by the properties of conditional expectation we have 

EZ{t*,t;) = sup E{E[Z{n,T;)\TrA} = sup EJ{n), 
TiGT neT 

where 

J {s)=E[Z{s,t;)\Ts]= ess snpE[Z{s,T2)\Ts]. (4) 

Therefore in order to find rj* and r|, we have to calculate J(s) first. The 
process J(s) corresponds to the value of revenue function in the one stopping 
problem if the observation starts at the moment s. 



3 Construction of the optimal second stopping time 

In this section, we will find the solution of one stopping problem defined 
by ([1]). We will first solve the problem for fixed number of claims, next we 
will consider the case with infinite number of claims. In this section we fix 
s - the moment when the change took place and m = Ms - the mass of the 
fishes at the time s. 



3.1 Fixed number of claims 



In this subsection we are looking for optimal stopping time T2q^ := T2 
such that 

E[Zis,TlK)\Ts] = esssnp E[Z{s,T2,k)\^s], (5) 

where s > is a fixed time when the position was changed and K is the 
maximum number of claims which can occur. Let us define 

eSSSUp E[Z{s,T2,n,K)\^s,7i] = E [Z{s,T2n K)\^s,n] , (6) 

T2,„,A-eT=_^, 
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for n = K,...,1,0 and observe that -T^^ = Z{s,T2.k)- The cr ucial role 
in our subsequent considerations plays the following lemma (see BremaudI 
( 198lh ). 

Lemma 1. //ti S T, T2 £ T", then there exist Ri^n S M{J^n) and R2,n G -M(-^s,n) 
respectively, such that Ti A Ti^„+i = (Ji.n + Ri,n) A Ti^n+i on {uj : Ti{uj) > 
« e {1,2}, a.s. 

Now we can derive the dynamic programming equations satisfied by j^- 
To simplify notation we write Mt = for t < s, M„ = Mti,„, = 
audi?, = 

Theorem 1. Lets>Q and Ff^j^ = Z(s,T2.k)- For n = K ~ 1, K ~ 2, . . . ,0 

we have 

r^K^ esssup 'dn,K{Ms,s,M^,T2,n,R2,n) a.s., 



where ■dn,K{m, s, m, t, r) = l{t<ta} yF2{r)\i{r<to-t]W2{m, s, m, t+r)-CI{r>to-t}] + 

E [l{S2,„+i<r}-r^+l,Kl-^s,r,] ^ - CI{t>to}. 



It can be shown that there exists i?2 „ such that F^^ ^ — k{Ms, s, M,^, r2,n , R2 n) 
for n < K — 1. 

Theorem 2. Let i?2 ^ &e t/ie sequence of Tg^i-measurahle random variables 
{fix i?2 ^ = 0) anrf ri!^^ ^ K A inf{i > n : R^^ < 52,^+1}, n = 0,.. .,K. 
Then F^ j^ = E [Z{s, T*,,^K)\^s,n] , where T*„ji = ^2,,,^ ^. + ^Lf^ ^- 

Lemma 2. F^ = l^^ln^M^, r2,n) /or n — . . . ,Q, where the sequence of 
functions 7|'™ is given recursively as follows: 

7o''"(m,i) = I{i<t„}W2(m, s,m,t) - Cl{t>to}, 

lj'"\m,t) = ^t<to}SnpK2^^--^{in,s,m,t,r) - Cl^tyto}, (7) 

r>0 ^ 

w/iere K2,sim, s,m,t,r) = F2(r)[I{^<t„_t}u;2(TO, s, m, i + r) - CI{r>to-t}] + 
/; dF2 (z) S{m + x,t + z)dH2 [x) . 

Let us denote ai — fi/Fi and let us set Ai{a) = E [gi{a + Xt) — gi{a)]. The 
sequence of functions 7''™ can be expressed as follows 



7j''"("i,t) = I{t<to}|ui2(m,s,m,t) +y2,j(m - m, t - s, to - t)| - C'I{t>t^} 



(8) 



where y2j{a,b, c) is given recursively as follows y2fi{a,b, c) — and y2j{a,b, c) — 
maxo<r<c 4>2,y2.j-i {a, b, c, r), 

(f)2,s{a,b,c,r) = / F2{z){a2{z)[A2{a) + E5{a + X2,b + z,c - z)] -C2{b + z)}dz. 
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The se cond optimal stopping time is constructed similarly like in iFerenstein and SierocinskH 
|l993). Let B = B([0,oo) X [0, to] X [0,to]) be the space of all bounded, continuous 
functions with the norm \\5\\ = sup^ j, ^ \5{a, h,c)\. It is complete space. Let us define 
the operator $2 '■ B ^ B as 

{$2S)(a,b,c) — max (f>2,s(a,b,c,r). (9) 

0<r<c 

We have 1/2, j (a, b, c) — (<?2j/2j-i)(a, b, c) and by ^ there exists a function (a, b, c) 

such that j/2j (a, b, c) = 02,j;2,3-i (^^i c, j(a, fo, c)) and {fh, t) = I{i<iQ} |w2(m, s, m, t) + 

4>2,y2 j-1 ("^ ~ m,t ~ s,to ~t, r2,j {fh — m, t — s, to — t)) j- — Cl^tx^}. The consequence 

of the foregoing considerations is the optimal stopping times r2,,j./f in following 
form: 

Theorem 3. Let R^i = rl^K-ii^i - Ms,T2,i - s,tQ - T2,i) for i = Q,1,...,K 
moreover rj^^K = K /\ inf{i > n : R2J < 52,^+1}, then the stopping time r2,„_K = 
12,77= K + -^2 ^ optimal in the class T^^fc '^^d F^j^ ~ E [^(s, T"2,n.if )l-^s,n] • 



3.2 Infinite number of claims 

The solution of one stopping problem, related to construction of the second stopping 
moment, for infinite number of claims is obtained under assumption that ^2(^0) < 1- 
If F2{to) < 1 then the operator $2 B B defined by (IS]) is a contraction. By 
((8| and the contraction properties we get by fixed point theorem that there exists 
2/2 G -B such that 3/2 = ^23/2 and limK^oo \\y2,K — 2/2!! = 0. It implies that 2/2 is 
measurable and 7"'™ — limK^oo Tk™ is given by 

7*''"(m,t) = l{t<to} [w2{m,s,m,t) + y2{fh ~ m,t - s,to - t)] - Cl{t>to}- 

We can now calculate the optimal strategy and the expected gain after changing 
place. 

Theorem 4. If F2{to) < 1 and has the density function f2, then 

(i) forn £ N the limit T2^„ = firnx—oo t"2 n.jf O'-S- exists andT2^j^ < to is an optimal 
stopping rule in the set T" D {t > T2.n} , 

(ii) E[Z{s,tI^)\Ts,„] =7='™(M;:,T2,„) a.s. 

It can be proved that the function "/"'"^{m, s) with respect to s is left-hand differen- 
tiable. It allows to construct the optimal the first optimal stopping moment similarly 
as the second one. To this end we have to take as the payoff function 7°''"(m, s) — 
I{s<to}"("^>s)-CI{s>to}, where u{m,s) = ffi(m)-ci(s)+p2(0)-C2(0)+j/2(to-s) is 
continuous, bounded, measurable with bounded left-hand derivatives with respect 
to s. The conditional value function of the second optimal stopping problem has 
the form: 

J{s) ^ E[Z{s,T;)\Ts]='y'''^'''{Ms,s) a.s. (10) 

The first optimal stopping moment in the considered problem is equal r* such that 
JK) =sup^g^i5J(r). 



Example 1. li S2 has exponential distribution with constant hazard rate Q2, g2 is 
increasing and concave, C2 is convex and t2,n = 72, n, tti^ = then 

r2*„ = m{{t € [t2,n, to] : a2[Eg2{m^ + X2 - m) - S'2(m^ - m)] < C2{t - s)}, 

where s is the moment of changing place. Moreover if S\ has exponential distribution 
with constant hazard rate ai, gi is increasing and concave, ci is convex and ti,„ = 
Ti^n, m„ = M„ then 

ri*„ = inf{s G [ti,n,to] : [Eg-i{;m„ + Xi) - (/i(-m„)] < ci(s)} 

If for i = 1 and i = 2 the functions grj are increasing and convex, a are concave and 
Si have exponential distribution with constant hazard rate a, then ri* „ = r|,„ = to 
for n e N. 

4 Conclusions 

This article presents the solution of double stopping problem in "fishing model" 
for finite horizon. The analytical properties of the reward function in one stopping 
problem played the crucial rule in our considerations and allowed us to extend the 
problem to double stopping. It is easy to generalize our model and the solution to 
multiple stopping problem. 
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